Method and apparatus for detecting camera sensor intensity saturation

ABSTRACT

A method for detecting intensity saturation of a light sensor includes monitoring an electrical signal from a light sensor for detecting an intensity saturation condition of at least one pixel of the light sensor, converting the electrical signal to a digital signal, imposing a reserved bit combination on the digital signal indicating the intensity saturation condition of the pixel, and transmitting a control signal in response to the digital signal to compensate for the intensity saturation condition of the pixel. Alternatively stated, the method includes monitoring pixel data from an output of a light sensor to determine a number of pixels at saturation and a number of pixels near saturation, comparing the number of pixels at saturation to a predetermined first threshold number, comparing the number of pixels near saturation to a predetermined second threshold number, reprogramming the light sensor to adapt to more brightness if the number of pixels at saturation is above the first threshold number, and reprogramming the light sensor to adapt to less brightness if the number of pixels near saturation is below the second threshold number. An apparatus for detecting intensity saturation of a light sensor includes a saturation detector for detecting and measuring an intensity saturation condition of at least one pixel of a light sensor, the intensity saturation condition of the pixel being at saturation upon receiving light with an intensity above a predetermined level, the saturation detector emitting a digital signal with a reserved bit combination indicating the intensity saturation condition of the pixel, and a processor receiving and processing the digital signal from the saturation detector and transmitting a control signal in response to the digital signal to compensate for the intensity saturation condition of the pixel.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 09/387,068, filedAug. 30, 1999 now U.S. Pat. No. 6,816,200, incorporated herein byreference, which claims the benefit of U.S. Ser. No. 60/098,581, filedAug. 31, 1998, also incorporated herein by reference.

BACKGROUND OF THE INVENTION

This invention relates generally to color digital cameras, and moreparticularly to the processing of pixel information generated by thesensor of a color digital camera.

Digital color cameras are used with computer or other digital processingsystems. Such cameras include a sensor, optics, preprocessingelectronics, and a cable or other communication link to transfer data tothe digital processing system. Digital cameras are made by Connectix,Intel, and others.

In a conventional digital camera, the sensor is often a charge coupleddevice (CCD) that produces electrical image signals corresponding to anobject producing or reflecting light onto the sensor. The electricalimage signals are then processed and recorded on a storage medium suchas a memory card or other computer readable medium.

A problem encountered with digital cameras is that the sensor can becomesaturated when the light intensity exceeds the intensity saturationlimit of the sensor. When the light level is above the saturation levelof the sensor, all further video information, other than the saturationinformation, is lost. In the past, this problem has been addressed byattempting to manipulate the amount of light striking the sensor oradjusting the sensor itself.

Some of the prior art has attempted to utilize a light intensitymeasuring circuit which processes data from the sensor and controlsmovement of an iris. The iris closes to restrict the amount of incidentlight striking the sensor and opens to allow more incident light tostrike the sensor. However, the movement of the iris is slow compared toother means of intensity adjustment. Furthermore, due to its mechanicalnature, the iris often closes more than necessary or doesn't closeenough, at which point it must be repositioned. The result is slowcorrection time and fluctuations in brightness, ending in degradation ofpicture quality. The latter is especially if utilized with videocameras, where the fluctuating brightness is captured. Furthermore, asolitary bright light can cause the iris to close so much that gradationof the dark regions becomes compressed and deteriorated. The same occurswhen direct strong light is incident, such as in the case of strong rearlighting.

In a system in which an electric charge corresponding to an amount oflight received is accumulated on a photodiode and passed to an n-layersubstrate of the sensor, other prior art has sought to draw away excessvoltage that flows from the photodiode of the sensor when an excessiveamount of light is received. For example, a p-layer is positionedbetween the photodiode and substrate of the sensor and grounded. Thisapplies a reverse bias voltage to the substrate and p-layer so that adepletion layer is formed between the photodiode and the substrate.Surplus electric charges that overflow from the photodiode, due to anexcessive amount of received light, are absorbed in the depletion layer.The voltage of the substrate is then adjusted to accommodate bright anddark scenes. A great disadvantage is that signals from the photodiodesare disrupted in that they must now pass through a layer specificallydesigned to absorb such signals. Another disadvantage of this is thatthe voltage of the substrate must be preset based on estimated lightconditions, especially disadvantageous for capturing moving video.Another disadvantage is that the voltage of the entire substrate must bechanged, not just for the portion receiving the excessively intenselight. This results in poor picture quality in that bright areas oflight are compensated for but dim spots are not.

Another problem encountered with digital cameras is that picture qualitydrops as the light intensity falls below a certain level. Furthermore,even images in a dark portion of a scene can be hard to observe if adirect strong light is incident in the scene. The prior art hasattempted to correct such problems by monitoring signals from the sensorthat have been separated into red, green and blue component signals andhave also been gamma corrected. Then, a dark area proportion in a wholepicked up image is detected. Next, a portion of the processed originalsignal is modified to stretch the dark signal region to improve thegradation of the dark area. The processed original signal and themodified signal are combined to output a resultant gradation improvedsignal. The disadvantage of this prior art method is that separation andgamma correction of the original signal are performed before the darkarea is detected. This reduces the accuracy of the detection of the darkarea. Further, the dark area is estimated from the processed originalsignal as a whole, not on a pixel by pixel basis, further reducing theaccuracy of the detection of the dark area and making correction of thedark area more difficult.

SUMMARY OF THE INVENTION

In the present invention, an electrical signal from a light sensor ismonitored to detect an intensity saturation condition of at least onepixel of the light sensor. The intensity saturation condition of thepixel is at saturation upon receiving light with an intensity above apredetermined level and below saturation upon receiving light with anintensity below a predetermined level. The electrical signal isconverted to a digital signal. A reserved bit combination is imposed onthe digital signal indicating the intensity saturation condition of thepixel. A control signal is transmitted in response to the bitcombination of the digital signal to compensate for the intensitysaturation condition of the pixel. An analog to digital converter can beutilized to convert the electrical signal to the digital signal. Theanalog to digital converter may be programmable to receive electricalsignals of different intensities from the light sensor.

Monitoring the electrical signal may include determining whether avoltage of an electrical signal from the light sensor is above apredetermined level. Further, the control signal may reset thepredetermined level of voltage. The analog to digital converter may formpart of an analog to digital circuit that also includes an AND gate andan OR gate.

The electrical signal from the light sensor may include a series ofsignals scanned from preselected pixels from an array of pixels of thesensor, or may include a series of signals scanned pixel by pixel fromthe array of pixels of the sensor.

In one aspect of the invention, an integrated circuit may perform theaforementioned actions. The integrated circuit may form part of a cameramodule. Optionally, the camera module may further include the lightsensor, a lens assembly aligned with the light sensor, and a printedcircuit board supporting the integrated circuit.

In another embodiment of the present invention, pixel data from anoutput of a light sensor is monitored to determine a number of pixels atsaturation. The number of pixels at saturation are compared to apredetermined threshold number. The light sensor is reprogrammed toadapt to more brightness if the number of pixels at saturation is abovethe threshold number. The light sensor may also be programmed to adaptto less brightness if the number of pixels at saturation is below thethreshold number.

A saturation detector may be coupled to the light sensor for detectingan intensity saturation condition of the light sensor. The saturationdetector can be reprogrammed to adapt to more brightness if the numberof pixels at saturation is above the threshold number. The monitoring ofthe pixel data may be performed without disturbing data flow.Optionally, reprogramming of the saturation detector can be performed inpredetermined increments.

In yet another embodiment of the present invention, pixel data from anoutput of a light sensor is monitored to determine a number of pixelsnear saturation. The number of pixels near saturation are compared to apredetermined threshold number. The light sensor is reprogrammed toadapt to less brightness if the number of pixels near saturation isbelow the threshold number.

A saturation detector may be coupled to the light sensor to detect anintensity saturation condition of the light sensor. The saturationdetector may be reprogrammed to adapt to less brightness if the numberof pixels at saturation is below the threshold number. Optionally,reprogramming of the saturation detector may be performed inpredetermined increments. The pixel data may also be monitored withoutdisturbing data flow.

In still yet another embodiment of the present invention, pixel datafrom an output of a light sensor is monitored to determine a number ofpixels at saturation and a number of pixels near saturation. The numberof pixels at saturation is compared to a predetermined first thresholdnumber and the number of pixels near saturation are compared to apredetermined second threshold number. The light sensor is reprogrammedto adapt to more brightness if the number of pixels at saturation isabove the first threshold number. The light sensor is reprogrammed toadapt to less brightness if the number of pixels near saturation isbelow the second threshold number.

In one aspect of the invention, a saturation detector may be coupled tothe light sensor to detect an intensity saturation condition of thelight sensor. The saturation detector can be reprogrammed to adapt tomore brightness if the number of pixels at saturation is above the firstthreshold number, and can be reprogrammed to adapt to less brightness ifthe number of pixels near saturation is below the second thresholdnumber. Optionally, reprogramming of the saturation detector can beperformed in predetermined increments. Also, monitoring the pixel datamay be performed without disturbing data flow.

In yet another embodiment of the present invention, a saturationdetector detects and measures an intensity saturation condition of atleast one pixel of a light sensor. The saturation detector emits adigital signal with a reserved bit combination indicating the intensitysaturation condition of the pixel. A processor receives and processesthe digital signal from the saturation detector and transmits a controlsignal in response to the digital signal to compensate for the intensitysaturation condition of the pixel.

The saturation detector may include a voltage detector for determiningwhether a voltage of an electrical signal from the light sensor is abovea predetermined level. The voltage detector may be programmable suchthat the control signal resets the predetermined level of the voltage.The saturation detector may include an analog to digital converter forconverting an electrical signal from the light sensor into a digitalsignal for indicating the intensity saturation condition of the pixel.Optionally, the analog to digital converter may be programmable toconvert electrical signals of different intensities. Also, the analog todigital converter may form part of an analog to digital circuit thatalso includes an AND gate and an OR gate

The present invention detects situations where too much light is beingreceived, or too little light is being received. When such conditionsare detected, they may be corrected by altering the setting of an ADCand a voltage detector on a preprocessor, and by altering the gammacorrection and color correction on a host or digital processor. Suchgamma and color correction can take place on a pixel by pixel basis.

An intensity saturation condition for each of the pixels of the lightsensor may be detected, and a reserved bit combination may be imposed onan ADC output bus to inform a digital processor of the saturationcondition for some or each of the independent pixels. A digitalprocessor can use this information to control the ADC to compensate forthe saturation condition.

Because the saturation condition of individual pixels is scanned, gammaand color correction may take place on a pixel-by-pixel basis, providingthe advantage of producing excellent picture quality free of excessivelybright and dim areas. Further, random sampling may be used to reduce theconsumption of electric power.

These and other advantages of the present invention will become apparentto those skilled in the art upon a reading of the following descriptionsof the invention and a study of the several figures of the drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a camera system including an apparatus fordetecting camera sensor intensity saturation;

FIG. 2 is a schematic of the camera sensor intensity saturationapparatus of FIG. 1;

FIGS. 3 a and 3 b shows a waveform illustrating operation of the voltagedetector shown in FIG. 2;

FIG. 4 is a table of the logic of the XOR gate;

FIG. 5 is a flow diagram for a process implemented by the digitalprocessor of FIG. 1 to control the ADC of FIG. 2 in response to thedetection of an intensity saturation condition of the sensor; and

FIG. 6 is a diagram illustrating the brightness value ranges associatedwith pixel data from a camera sensor.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In FIG. 1, a digital video camera system 10 includes a lens assembly 12,a sensor 14 for converting incident light into a video signal, ananalog/digital preprocessor 16, and a digital processor 18. The lensassembly 12 focuses an image of an object 20 on a surface 22 of thesensor 14. Often, the lens assembly 12, sensor 14, and preprocessor 16are housed within a common housing or enclosure (not shown). The digitalprocessor 18 may, in some circumstances, also be housed within the sameenclosure. However, in many instances the digital processor 18 can be anexternal processor, such as a microcomputer.

The lens assembly can be in any suitable form, including a simpleplastic lens assembly or a complex multi-lens optical assembly,depending upon the application. The sensor 14 is one of a variety ofsolid-state sensors made by, for example, charge coupled device (CCD) orcomplementary metal oxide semiconductor (CMOS) technologies.

The preprocessor 16 preferably includes a number of sections includingan analog to digital converter (ADC) 24, other processing circuitry 26,and control logic 28. Analog sensor data is input into the preprocessor16 over an analog bus 30, and digital sensor data is output from thepreprocessor 16 over digital bus 32. The digital processor 18 receivesthe digital “video” data over bus 32, and provides control signals tothe preprocessor 16 over control bus 34. As will be discussed in greaterdetail below, the preprocessor 16 of the present invention detects anintensity saturation condition on the analog “video” bus 30, informs thedigital processor 18 of the condition by a particular bit pattern overthe digital video bus 32, and then the digital processor 18 providesappropriate control signals over the control bus 34 to compensate forthe saturation condition.

In FIG. 2, the ADC circuit 24 of the present invention is shown ingreater detail. More particularly, one embodiment of ADC circuit 24includes a programmable ADC 36, a programmable voltage detector 38, anAND gate 40, and an exclusive or (XOR) gate 42. The ADC in the presentexample receives an analog input on a line 44 and has ten (10) outputlines 46 as the ADC output. These outputs range from the mostsignificant bit (msb) to the least significant bit (lsb). The circuitryand logic of the ADC circuitry 24 imposes a particular bit pattern onthe preprocessed ADC output 48 when the intensity level on the line 44exceeds a predetermined value or threshold, or is otherwise detected tobe in saturation. The particular bit pattern chosen in this embodimentis “1111111111” in binary or “1023” in decimal.

The ADC 36 is a conventional programmable ADC circuit that takes theanalog signal on line 44 and converts it to digital signal on bus 48.Therefore, this ADC has ten bits of resolution. Other embodiments havemore, none or fewer bits of resolution. The signal on line 46 is aseries of signals scanned pixel-by-pixel from the array of pixels on thesurface 22 of sensor 14. The ADC 36 can be controlled as far assensitivity to different intensity ranges via an ADC control line 50forming a part of the control bus 36 from the digital processor 18.

The programmable voltage detector 38 is used to detect the voltage levelon line 44 to determine when a particular pixel is saturated. When thepixel is saturated, the ADC output is set to all 1's to signal asaturation condition for that pixel. It should therefore be noted thatthe saturation detection is on a pixel-by-pixel basis, and that thedigital processor 18 can act upon the pixel saturation information tocontrol the ADC via ADC control line 50.

The actual threshold V_(TH) detected by the detector 38 can be set by adetector program line 52. More particularly, the detector 38 can includea register that can be programmed via line 52 to determine the thresholdvoltage V_(TH) on line 44 which will create a digital output signal on aline 54. In the present embodiment, the digital output signal on line 54is “1” or “HI” when the voltage V on the line 44 is below V_(TH), andthe digital output signal on line 54 is “0” or “LO” when the voltage Von the line 44 is above V_(TH). Therefore, in this embodiment, a LOsignal on the line 54 indicates a saturation or “whiteout” condition.FIG. 3 illustrates the relationship between the voltage V on line 44 andthe output of the detector 38. Of course, other circuits can be used todetermine a saturation condition for the sensor array. Voltage thresholddetection is only one method for accomplishing this task.

The AND gate 40 has 11 inputs, where one input is coupled to the outputline 54 of the detector 38, and where the remaining 10 inputs arecoupled to the ADC output bus 46. Therefore, the output of the AND gate40 on a line B will be “1” or “HI” only when the bus 46 is at 1111111111(in binary) and when the voltage V on input line 44 is below V_(TH).Under these conditions, the output X of XOR 42 is forced to 0 (LO),forcing the output of preprocessed ADC output bus 48 to 111111110. Thisis because the output on the bus 48 of 1111111111 is reserved to signala saturation condition.

When the voltage V on line 44 is above the threshold voltage V_(TH), theoutput of the ADC on bus 46 will be 1111111111, and the signal on line54 will be 0. This will cause the output of the AND gate 40 to go “LO”or “0”, allowing the output of the XOR 42 to become the same as the lsbon bus 46, i.e. to become 1. The output bus 48 will therefore be1111111111, signaling the saturation of that pixel. When the output onthe bus 46 is other than 1111111111, the output of the AND gate 40 willbe 0, and the data on the bus 48 will be the same as the data on the bus46.

The logic of the XOR gate 42 is shown in FIG. 4. That is, when the inputon line B to the XOR gate 42 is LO, then the output of the XOR gate isthe same as the input on line A. The condition where A is LO and B is HIis impossible, since a LO signal on A would force B LO as well. Whenboth A and B are both HI, then x is forced LO by the XOR gate 42.

In FIG. 5, a process 56 for detecting camera sensor intensity saturationas implemented by a digital processor 18 begins at 58, and the flags “A”and “B” are cleared in an operation 60. Then, in an operation 62, theprocess “eavesdrops” on a frame of pixels; keeping a running count ofbright and whiteout pixels. By “eavesdrops” it is meant that the processdoes not disturb the data flow, but merely monitors the pixel data as itis received. This is usually accomplished by using a few registers tokeep count of certain pixel types, including bright and whiteout pixels.A “bright” pixel is near, but not yet at, saturation. A “whiteout” pixelis at saturation, and is designated by “1111111111” in this example.

Next, in a decision operation 64, it is determined whether the number ofwhiteout pixels are greater than a threshold1 number. A typicalthreshold1 number for whiteout pixels depends upon the brightness of ascene, but for an average scene, the threshold1 may be about 20 whiteoutpixels. If it is greater than the threshold1, an operation 66 sets flagA to indicate that the preprocessor 16 should be reprogrammed for abrighter scene. This is accomplished, at least in part, by reprogrammingthe ADC 50 and the voltage detector 38.

Operation 68 determines whether the number of bright pixels is less thana threshold2, which is typically about 50, but which again is dependentupon the scene. If it is less than the threshold, operation 70 sets flatB to flag that the preprocessor 16 should be reprogrammed for a darkerscene. This, again, is accomplished in part by reprogramming the ADC 50and the voltage detector 38.

Operation 72 determines whether the A flag has been set. If so,operation 74 reprograms the ADC 50 and the voltage detector 38 to handlean incrementally brighter scene. The ADC 50 is programmed to cover awider range of voltages with the same number of bit combinations. Thevoltage detector 38 is reprogrammed to raise the threshold voltageV_(TH). This is accomplished incrementally to avoid making too large ofa step.

Operation 76 determines whether the B flag is set and the A flag is notset. If these conditions are true, then an operation 78 reprograms thesensor ADC 36 and the voltage detector 38. More particularly, the ADC 50is programmed to cover a narrower range of voltages for the same numberof bit combinations. The voltage detector 38 is reprogrammed to lowerthe threshold voltage V_(TH). This is accomplished incrementally toavoid making too large of a step. The process 56 is then completed at 80awaiting the next frame of data.

FIG. 6 illustrates the different categories of brightness in the presentexample. Here, the value 1023 represents whiteout, while values of about900 to 1022 represent the bright pixel range. Values below about 900represent the non-bright pixel range.

It will therefore be appreciated that the process and apparatus of thepresent invention detects a saturation condition and forces a digitaloutput bus to a predetermined pattern to indicate the saturationcondition. The process and apparatus of the present invention preventsthe predetermined pattern from occurring when the saturation conditionis not present.

It will further be appreciated that the present invention detectssituations where too much light is being received, or too little lightis being received. When such conditions are detected, they are correctedby altering the setting of an ADC and a voltage detector on apreprocessor, and by altering the gamma correction and color correctionon a host or digital processor, as will be appreciated by those skilledin the art.

It should be noted that the logic used in the example of FIGS. 1 and 2and the processes illustrated in FIG. 5 exemplify only oneimplementation of the present invention. Those skilled in the art willappreciate that other logic components can accomplish the design goalsof the present invention. For example, NAND gates, OR gates, invertersand other forms of logic can be used to accomplish the goals of theinvention, which includes the detection of a saturation condition, theimposition of a particular bit pattern on the video output bus, and thecorrection of the saturation condition in response thereto. Likewise,alternative process operations can be used to implement the presentinvention.

1. A digital camera device comprising: a multi-pixel light sensor; ananalog-to-digital converter circuit coupled to an output of saidmulti-pixel light sensor; and a lighting condition adjustment circuitcoupled to an output of said analog-to-digital converter and operativeto adjust an operating parameter of said analog-to-digital converter tocompensate for detected lighting conditions; wherein said lightingcondition adjustment circuit counts a number of digital outputs of saidanalog-to-digital converter which satisfy a condition and creates anincremental analog-to-digital converter control signal to be applied tosaid analog-to-digital converter to adjust said operating parameter. 2.A digital camera device as recited in claim 1 wherein said condition issatisfied by at least one of being above a threshold value and equalingsaid threshold value.
 3. A digital camera device as recited in claim 1wherein said condition is satisfied by at least one of being below athreshold value and equaling said threshold value.
 4. A digital cameradevice as recited in claim 3 wherein said threshold value is a firstthreshold value, and said condition is further satisfied by at least oneof being above a second threshold value and equaling said secondthreshold value.
 5. A digital camera device as recited in claim 1further comprising a lens associated with said multi-pixel light sensor.6. A digital camera device as recited in claim 5 further comprising adisplay for displaying a representation of an image focused on saidmulti-pixel light sensor by said lens.
 7. A digital camera device asrecited in claim 5 further comprising a communication link fortransmitting a representation of an image focused on said multi-pixellight sensor by said lens.
 8. A digital camera device as recited inclaim 5 further comprising a storage medium for storing a representationof an image focused on said multi-pixel light sensor by said lens.